COVID-19 response: effectiveness of weekly rapid risk assessments, Italy

Abstract Problem After Italy’s first national restriction measures in 2020, a robust approach was needed to monitor the emerging epidemic of coronavirus disease 2019 (COVID-19) at subnational level and provide data to inform the strengthening or easing of epidemic control measures. Approach We adapted the European Centre for Disease Prevention and Control rapid risk assessment tool by including quantitative and qualitative indicators from existing national surveillance systems. We defined COVID-19 risk as a combination of the probability of uncontrolled transmission of severe acute respiratory syndrome coronavirus 2 and of an unsustainable impact of COVID-19 cases on hospital services, adjusted in relation to the health system’s resilience. The monitoring system was implemented with no additional cost in May 2020. Local setting The infectious diseases surveillance system in Italy uses consistent data collection methods across the country’s decentralized regions and autonomous provinces. Relevant changes Weekly risk assessments using this approach were sustainable in monitoring the epidemic at regional level from 4 May 2020 to 24 September 2021. The tool provided reliable assessments of when and where a rapid increase in demand for health-care services would occur if control or mitigation measures were not increased in the following 3 weeks. Lessons learnt Although the system worked well, framing the risk assessment tool in a legal decree hampered its flexibility, as indicators could not be changed without changing the law. The relative complexity of the tool, the impossibility of real-time validation and its use for the definition of restrictions posed communication challenges.


Introduction
In its first response to the emerging epidemic of coronavirus disease 2019 (COVID-19), the Italian government implemented national restriction measures with strict physical distancing and restrictions on public movements 1 that were maintained from March to May 2020. 2 Although followed by a rapid decrease in transmission of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), 1 the control measures had an adverse impact on the economy and society. 3 To continue and further improve Italy's preparedness and response capacity, 4 we used an adapted version of the European Centre for Disease Prevention and Control (ECDC) rapid risk assessment tool 5,6 to conduct weekly subnational assessments using data from public health intelligence.

Local setting
Italy's health-care system is decentralized at regional level across 21 regions and autonomous provinces that largely differ in terms of size, population density and age structure as well as the financing and delivery of health care. 7 The infectious diseases surveillance system uses consistent data collection methods across the country, with local health units collecting notifications from clinical services and investigating cases.
Local health units transfer the data to regional and autonomous provincial authorities who verify and notify cases to the Ministry of Health and to the Italian national institute of health (Istituto Superiore di Sanità; the national centre for research, control and scientific advice on public health).
We designed the Italian COVID-19 risk assessment tool to provide reliable assessments in the different regional contexts. The risk assessments produced for each region and autonomous province were initially used to informally assist regional epidemic responses. Subsequently, in October 2020 the assessments were formally integrated as part of a flexible COVID-19 prevention and control response strategy for the autumn to winter 2020 season. 4 From the beginning of November 2020, 8 the Italian government implemented laws to define the need for strengthening or easing of regional interventions to control the epidemic, according to fixed parameters. Until May 2021, the weekly regional risk assessments became one of the parameters used to define the level of restrictions to be implemented. We have shown the effectiveness of this approach in a previous publication. 9 Higher immunization coverage has decreased the impact of COVID-19 in terms of severe disease and death, and therefore the legal parameters used for mitigation of the pandemic were changed and risk assessments were no longer used. However, these assessments are still produced to date Problem After Italy's first national restriction measures in 2020, a robust approach was needed to monitor the emerging epidemic of coronavirus disease 2019 (COVID-19) at subnational level and provide data to inform the strengthening or easing of epidemic control measures. Approach We adapted the European Centre for Disease Prevention and Control rapid risk assessment tool by including quantitative and qualitative indicators from existing national surveillance systems. We defined COVID-19 risk as a combination of the probability of uncontrolled transmission of severe acute respiratory syndrome coronavirus 2 and of an unsustainable impact of COVID-19 cases on hospital services, adjusted in relation to the health system's resilience. The monitoring system was implemented with no additional cost in May 2020. Local setting The infectious diseases surveillance system in Italy uses consistent data collection methods across the country's decentralized regions and autonomous provinces. Relevant changes Weekly risk assessments using this approach were sustainable in monitoring the epidemic at regional level from 4 May 2020 to 24 September 2021. The tool provided reliable assessments of when and where a rapid increase in demand for health-care services would occur if control or mitigation measures were not increased in the following 3 weeks. Lessons learnt Although the system worked well, framing the risk assessment tool in a legal decree hampered its flexibility, as indicators could not be changed without changing the law. The relative complexity of the tool, the impossibility of real-time validation and its use for the definition of restrictions posed communication challenges.
Lessons from the field Data for action in COVID-19 response, Italy Flavia Riccardo et al.
with the same method to monitor the epidemic and are published online. 10

Approach
The aim of the risk assessment was to provide a weekly overall categorization of the risk of an uncontrolled and unsustainable SARS-CoV-2 outbreak in each of the 21 Italian regions and autonomous provinces. As per ECDC guidance, we defined risk as a combination of the probability of the health threat and its impact, 5,6 described using consolidated risk categories (low, moderate, moderate with high probability of evolving to high, and high). We first assessed the probability and impact separately and then combined them to provide an overall risk, which we then adjusted in relation to an assessment of the health system's resilience. This resilience element was an innovation not included in the original ECDC framework (see the authors' data repository). 11 The three components of the Italian risk assessment tool therefore were: (i) probability (evidence of increased transmission, as a proxy for increasing probability of uncontrolled SARS-CoV-2 transmission in a region or autonomous province); (ii) impact (evidence of unsustainable burden of COVID-19 cases on hospital services); and (iii) resilience (capacity of the public health system to withstand the burden of the pandemic and maintain its functions within the test, track and trace strategy).
We used three sets of quantitative indicators to assess these components (Table 1). 12 The first set (data quality output indicators) defined the minimum levels of data completeness to allow the risk assessment; the second set (resilience output indicators) monitored the resilience of public health services in maintaining high levels of testing and contact tracing; the third set (probability and impact result indicators) monitored the probability of an uncontrolled spread of SARS-CoV-2 and of an unsustainable impact of COVID-19 on hospital services. For each indicator, we defined thresholds for alerts.
We assessed probability and impact separately using two dedicated algorithms each composed of three trigger questions requiring a yes/no answer. The first two questions in each algorithm were quantitative while the last question was qualitative. 11 We compiled the quantitative questions using data from the described indicators (Table 1). To answer the qualitative questions, we activated the national event-based surveillance system 13 and received weekly declarations from regional public health authorities. More specifically, regional authorities declared respectively if an uncontrolled SARS-CoV-2 transmission that could not be managed locally, or if new clusters of infection in vulnerable settings, were occurring. 11 Once the assessment of the two algorithms for probability and impact was concluded, a first risk level was defined. We then looked at the resilience indicators. If we detected multiple alerts from the resilience indicators, we automatically scaled up the initial risk to the next risk level.

Relevant changes
Between 4 May 2020 and 24 September 2021, the Italian national institute of health performed 71 weekly risk assessments. 10 Each assessment reported an updated classification of risk for each Italian region or autonomous province. As shown in Fig. 1, the risk assessments captured regional risk heterogeneities and were consistent overall with the national epidemic curve. In the data repository, 11 graphs illustrate how the level of risk assigned was accurate in signalling when increases in the incidence of laboratory-confirmed severe and lethal infections were expected to occur within 3 weeks in the absence of additional control or mitigation measures.
In its early implementation, during low viral circulation in spring to summer 2020, the risk assessment system was very sensitive to localized clusters with limited cases, especially in smaller regions or autonomous provinces. The risk assessment therefore changed occasionally from low to moderate and then back to low as the clusters were contained. These findings, although consistent with the data, were misinterpreted as false alarms and led to some initial concern and distrust in the method among subject-matter experts. As the indicators could not be changed without changing the law, we solved these initial issues by clarifying concerns with public health officials without modifying the risk assessment tool.
Subsequently, the perceived complexity of the tool and the fact that risk assessments always addressed the previous week (too delayed) were criticized. 14 The net reproduction number (Rt), which reflects the transmissibility of the disease, was also debated and for similar reasons. Even though the risk assessment tool only gave a very limited weight to Rt (Table 1), this controversy targeted its overall validity.
The weekly publication of the risk assessment findings 10 became a contested topic in the media, 14 increasingly so between November 2020 and May 2021, when higher risk was automatically associated by law with the enforcement of more severe restrictions to control the spread of the virus. Especially during the autumn to winter 2020 peak of COVID-19 cases, criticism of the assessment system expressed through the media increased, and numerous legal actions were started by representatives of different interest groups and organizations. However, to date, none of the legal actions have led to a re-evaluation of the published risks. Strategies that we adopted to improve public understanding included a weekly presentation of the risk assessments in a press conference and the production of releases and frequently asked questions pages on institutional websites. 15 The assessment method became less debated after its automatic impact on decision-making stopped.

Lessons learnt
During a protracted outbreak, ensuring that control measures against the spread of disease are proportionate to the risk is important to limit an unwarranted impact on the economy and on the overall well-being of the population. Ensuring accountability and transparency to the general population is also needed. The risk assessment system supported decision-making in Italy by effectively anticipating when the disease outbreak was expected to rapidly worsen, harnessing existing data flows at national and subnational level. The system operated without dedicated funding but, despite requiring a large amount of staff time, was sustainable in the medium term without any disruption in the weekly production of updated risk assessment reports.
Continuous communication between the experts at the national institute of health and public health officials across all the regions and autonomous provinces made sure that the assessments reflected what was happening locally each week while supporting the Data for action in COVID-19 response, Italy Flavia Riccardo et al.   Data for action in COVID-19 response, Italy Flavia Riccardo et al.
cohesion of the public health network. Also, different data sources and multiple indicators were helpful in maintaining the robustness of the assessment during increased transmission.
The main challenge in conducting the risk assessments during the pandemic was related not to the method's performance but to difficulties in public communication. While we addressed initial misunderstandings among public health officers through technical discussions, the situation changed when the risk assessments started directly impacting restrictions and, consequently, peoples' daily lives and livelihoods. Criticism of the risk assessment tool (too complex) or of specific parameters (too delayed) stopped being a technical discussion among subject-matter experts and became a contested topic for decision-makers and the general public.
A similar approach to capturing the components of probability, impact and resilience could be adopted in different countries by adapting existing data flows and deploying available human resources. However, we learnt that to increase the acceptability of risk assessment tools like the one described, robustness in performance is not enough (Box 1). Communication issues in applying risk assessment tools should be anticipated during an emergency. Approaches to resolve these issues need to be designed with communication experts, alongside the development of the risk assessment tools, to ensure that the acceptability of the risk assessments is not damaged by controversies driven by misunderstandings. ■
Marco regional El sistema de vigilancia de las enfermedades infecciosas en Italia aplica métodos de recopilación de datos coherentes en todas las regiones y provincias autónomas descentralizadas del país. Cambios importantes Las valoraciones semanales de los riesgos mediante este enfoque fueron sostenibles en la supervisión de la epidemia a nivel regional entre el 4 de mayo de 2020 y el 24 de septiembre de 2021. La herramienta proporcionó valoraciones fiables de cuándo y dónde se produciría un rápido aumento de la demanda de servicios sanitarios si no se incrementaban las medidas de contención o mitigación en las tres semanas siguientes. Lecciones aprendidas Aunque el sistema funcionó bien, el hecho de enmarcar la herramienta de valoración de los riesgos en un decreto legal dificultó su flexibilidad, ya que los indicadores no se podían modificar sin cambiar la ley. La relativa complejidad de la herramienta, la imposibilidad de validación en tiempo real y su uso para la definición de las restricciones plantearon problemas de comunicación.